PART-OF-SPEECH TAGGING

COMBO: State-of-the-Art Morphosyntactic Analysis (2021.emnlp-demo)

Challenge:	Existing tools that teach an independent model for each task are not supported in Chinese.
Approach:	They propose an open-source neural language platform supporting six Chinese NLP tasks . source code, documentation, and pre-trained models are available at https://github.com/hit-SCIR/ltp .
Outcome:	The proposed platform supports six Chinese NLP tasks.

Copied to clipboard

Mateusz Klimaszewski, Alina Wróblewska

At a Glance: The Impact of Gaze Aggregation Views on Syntactic Tagging (D19-64)

Challenge:	COMBO is an end-to-end NLP system for accurate part-of-speech tagging, morphological analysis, and (enhanced) dependency parsing.
Approach:	They propose a fully neural NLP system for accurate part-of-speech tagging, morphological analysis, lemmatisation, and (enhanced) dependency parsing.
Outcome:	The proposed system predicts categorical morphosyntactic features whilst also exposes their vector representations, extracted from hidden layers.

Copied to clipboard

Sigrid Klerke, Barbara Plank

Trankit: A Light-Weight Transformer-based Toolkit for Multilingual Natural Language Processing (2021.eacl-demos)

Challenge:	Recent work uses gaze data at the type level or at the token level and mostly from a single eye-tracking corpus.
Approach:	They propose to use gaze data to capture central tendency or variability of gaze data and to integrate binary phrase chunking and part-of-speech tagging.
Outcome:	The proposed approaches capture the central tendency or variability of gaze data better than proposed local views which retain individual participant information.

Copied to clipboard

Minh Van Nguyen, Viet Dac Lai, Amir Pouran Ben Veyseh, Thien Huu Nguyen

Converting the Point of View of Messages Spoken to Virtual Assistants (2020.findings-emnlp)

Challenge:	Trankit is a lightweight, pre-trained toolkit for multilingual natural language processing.
Approach:	They propose a transformer-based toolkit for multilingual natural language processing that trains pipelines over 100 languages and 90 pretrained pipelines for 56 languages.
Outcome:	The proposed tool outperforms existing pipelines over sentence segmentation, part-of-speech tagging, morphological feature tabbing, and dependency parsing while maintaining competitive performance over tokenization, multi-word token expansion, and lemmatization over 90 Universal Dependencies treebanks.

Copied to clipboard

Gunhee Lee, Vera Zu, Sai Srujana Buddi, Dennis Liang, Purva Kulkarni, Jack FitzGerald

Sequence Tagging with Contextual and Non-Contextual Subword Representations: A Multilingual Evaluation (P19-1)

Challenge:	Using a voice message, virtual assistants extract the message and send it to the user’s contact, rather than properly converting it to “I love you.”
Approach:	They propose to take a voice message from one user, convert it to “I love you” and deliver it to its target user.
Outcome:	The proposed system can take a voice message from one user, convert the point of view of the message, and then deliver the result to its target user.

Copied to clipboard

Benjamin Heinzerling, Michael Strube

Unleashing the True Potential of Sequence-to-Sequence Models for Sequence Tagging and Structure Parsing (2023.tacl-1)

Challenge:	Pretrained contextual and non-contextual subword embeddings are available in over 250 languages, allowing massively multilingual NLP.
Approach:	They compare pretrained contextual and non-contextual subword embeddings with a contextual representation method, namely BERT, on multilingual named entity recognition and part-of-speech tagging.
Outcome:	The proposed method outperforms non-contextual embeddings on multilingual named entity recognition and part-of-speech tagging.

Copied to clipboard

Han He, Jinho D. Choi

Unsupervised Boundary-Aware Language Model Pretraining for Chinese Sequence Labeling (2022.emnlp-main)

Challenge:	Sequence-to-Sequence (S2S) models have been successful on text generation tasks . however, learning complex structures with S2S models remains challenging .
Approach:	They propose to use constrained decoding to model part-of-speech tagging, named entity recognition, constituency, and dependency parsing tasks with 3 lexically diverse linearization schemas and corresponding constrained coding methods.
Outcome:	The proposed methods outperform the state-of-the-art on four core tasks.

Copied to clipboard

Peijie Jiang, Dingkun Long, Yanzhao Zhang, Pengjun Xie, Meishan Zhang, Min Zhang

Handling and Presenting Harmful Text in NLP Research (2022.findings-emnlp)

Challenge:	Experimental results show that Boundary-Aware BERT can improve Chinese sequence labeling tasks.
Approach:	They propose to encode boundary information directly into pre-trained language models . they propose to use unsupervised boundary information instead of supervised boundary info .
Outcome:	The proposed architecture improves Chinese sequence labeling tasks on ten benchmarks.

Copied to clipboard

Hannah Kirk, Abeba Birhane, Bertie Vidgen, Leon Derczynski

Towards Fine-grained Classification of Climate Change related Social Media Text (2022.acl-srw)

Challenge:	Text data can pose a risk of harm, but the risks remain unresolved in the NLP community.
Approach:	They propose an analytical framework categorising harms on three axes: harm type, whether harm sought as a feature of research design, whether harmful content is encountered when working on unrelated problems, and who it affects .
Outcome:	The proposed framework categorises harms on three axes: harm type, whether harm sought as feature of research design, and whether harmful content is encountered when working on unrelated problems.

Copied to clipboard

Roopal Vaid, Kartikey Pant, Manish Shrivastava

Information Extraction & Text Mining

Challenge:	a new study examines the fine-grained classification and classification of climate change-related social media text.
Approach:	They propose to use two datasets to analyze climate change-related social media text and propose a fine-grained classification based on the proposed dataset.
Outcome:	The proposed datasets are compared with existing datasets and benchmarked using the best-performing model.

Distilling the Knowledge of Romanian BERTs Using Multiple Teachers (2022.lrec-1)

Copied to clipboard

Andrei-Marius Avram, Darius Catrina, Dumitru-Clementin Cercel, Mihai Dascalu, Traian Rebedea, Vasile Pais, Dan Tufis

Re-framing Incremental Deep Language Models for Dialogue Processing with Multi-task Learning (2020.coling-main)

Challenge:	Existing approaches to train pre-trained language models focus on the English language, thus widening the gap when considering low-resource languages.
Approach:	They propose three versions of distilled BERT models for the Romanian language . they argue that the models offer performance comparable to their teachers .
Outcome:	The proposed models perform comparable to their teachers, while being twice as fast on a GPU and 35% smaller.

Copied to clipboard

Morteza Rohanian, Julian Hough

Efficient Test Time Adapter Ensembling for Low-resource Language Varieties (2021.findings-emnlp)

Challenge:	Using a multi-task learning framework, we train a universal incremental dialogue processing model with four tasks of disfluency detection, language modelling, part-of-speech tagging and utterance segmentation in a simple deep recurrent setting.
Approach:	They propose a multi-task learning framework to train a universal incremental dialogue processing model with four tasks of disfluency detection, language modelling, part-of-speech tagging and utterance segmentation in a simple deep recurrent setting.
Outcome:	The proposed model outperforms individual tasks and delivers competitive performance.

Copied to clipboard

Xinyi Wang, Yulia Tsvetkov, Sebastian Ruder, Graham Neubig

Multiple Tasks Integration: Tagging, Syntactic and Semantic Parsing as a Single Task (2021.eacl-main)

Challenge:	Specialized language and task adapters have been proposed to facilitate cross-lingual transfer of multilingual pretrained models.
Approach:	They propose a method that optimizes the ensemble weights of pretrained adapters for each test sentence by minimizing the entropy of its predictions.
Outcome:	The proposed method improves robustness to uncovered languages without training new adapters.

Copied to clipboard

Timothée Bernard

Predicting News Headline Popularity with Syntactic and Semantic Knowledge Using Multi-Task Learning (D18-1)

Challenge:	Existing systems that bypass intermediate levels of analysis are prone to error propagation and are therefore free from interference.
Approach:	They propose a multitask paradigm orthogonal to weight sharing that uses multiple tasks to process input iteratively but concurrently at multiple levels of analysis.
Outcome:	The proposed model uses reinforcement learning and release from sequential constraints to improve the quality of the syntactic and semantic parses.

Copied to clipboard

Sotiris Lamprinidis, Daniel Hardt, Dirk Hovy

Noise-Robust Morphological Disambiguation for Dialectal Arabic (N18-1)

Challenge:	Pre-trained word embeddings provide significant improvements over untrained embeddables . Feature analysis reveals structural patterns of headline popularity .
Approach:	They use a multi-task GRU network to model headline popularity . they find that pre-trained word embeddings provide significant improvements over untrained embeddables .
Outcome:	The proposed model improves on pre-trained word embeddings and untrained embeddables . it also improves with the combination of two auxiliary tasks, news-section prediction and part-of-speech tagging .

Copied to clipboard

Nasser Zalmout, Alexander Erdmann, Nizar Habash

Parsing Tweets into Universal Dependencies (N18-1)

Challenge:	Noisy content is non-canonical in nature, with lexical, orthographic, and phonetic variations.
Approach:	They propose a neural morphological tagging and disambiguation model for Egyptian Arabic with various extensions to handle noisy content.
Outcome:	The proposed model achieves about 5% relative error reduction over a state-of-the-art baseline for Egyptian Arabic.

Copied to clipboard

Yijia Liu, Yi Zhu, Wanxiang Che, Bing Qin, Nathan Schneider, Noah A. Smith

Mini But Mighty: Efficient Multilingual Pretraining with Linguistically-Informed Data Selection (2023.findings-eacl)

Challenge:	a new tweet treebank for English is designed to analyze tweets with universal dependencies (UD).
Approach:	They extend the universal dependencies guidelines to include special constructions in tweets that affect tokenization, part-of-speech tagging, and labeled dependencies.
Outcome:	The proposed method outperforms state-of-the-art parsers on other treebanks in accuracy and speed.

Copied to clipboard

Tolulope Ogunremi, Dan Jurafsky, Christopher Manning

Strong Baselines for Neural Semi-Supervised Learning under Domain Shift (P18-1)

Challenge:	AfriBERTa shows that training transformer models from scratch on 1GB of data from many unrelated African languages outperforms massively multilingual models on downstream NLP tasks.
Approach:	They propose that training on smaller amounts of data but from related languages could match the performance of models trained on large, unrelated data.
Outcome:	The proposed model outperforms models trained on large, unrelated datasets on downstream NLP tasks.

Copied to clipboard

Sebastian Ruder, Barbara Plank

Coding Textual Inputs Boosts the Accuracy of Neural Networks (2020.emnlp-main)

Challenge:	Existing neural models for learning under domain shifts only evaluate on a single task, on proprietary datasets, or compare to weak baselines.
Approach:	They propose a multi-task tri-training method that reduces time and space complexity of classic bootstrapping approaches.
Outcome:	The proposed method outperforms the state-of-the-art for sentiment analysis on two benchmarks.

Copied to clipboard

Abdul Rafae Khan, Jia Xu, Weiwei Sun

Challenge:	a new approach to natural language processing uses arbitrary symbols to represent meaning . Soundex, MetaPhone, NYSIIS, logogram are used as inputs for NLP .
Approach:	They propose to use arbitrary symbols to represent linguistic meaning of a word . they propose to integrate codewords with text to provide more reliable inputs .
Outcome:	The proposed approach outperforms state-of-the-art models on machine translation, language modeling, and part-of speech tagging.

Minimax and Neyman–Pearson Meta-Learning for Outlier Languages (2021.findings-acl)

Copied to clipboard

Edoardo Maria Ponti, Rahul Aralikatte, Disha Shrivastava, Siva Reddy, Anders Søgaard

Robustness in NLP

Challenge:	Model-agnostic meta-learning (MAML) is a strategy to learn resource-poor languages in a sample-efficient fashion.
Approach:	They propose a model-agnostic meta-learning strategy that minimizes the expected risk across languages with a uniform prior . they propose 'minimax' and 'neyman-pearson' models that constrain the risk in each language to a maximum threshold.
Outcome:	The proposed model reduces the maximum risk across languages while constraining the risk in each language to a maximum threshold.

Assessing Emoji Use in Modern Text Processing Tools (2021.acl-long)

Copied to clipboard

Abu Awal Md Shoeb, Gerard de Melo

Multimodality

Challenge:	Emojis are textual elements that are encoded as characters but rendered as small digital images or icons that can be used to express an idea or emotion.
Approach:	They propose to use a set of popular NLP tools to assess the support of emojis in tweets.
Outcome:	The proposed methods show that many systems still have notable shortcomings when operating on text containing emojis.

How much pretraining data do language models need to learn syntax? (2021.emnlp-main)

Copied to clipboard

Laura Pérez-Mayos, Miguel Ballesteros, Leo Wanner

Everything Is All It Takes: A Multipronged Strategy for Zero-Shot Cross-Lingual Information Extraction (2021.emnlp-main)

Challenge:	Pretraining methods are convenient, but expensive in terms of time and resources.
Approach:	They investigate the impact of pretraining data size on the syntactic capabilities of RoBERTa by using syntaktic structural probes to determine whether models pretrained on more data encode a higher amount of syntastic information.
Outcome:	The proposed models perform better on part-of-speech tagging, dependency parsing and paraphrase identification.

Copied to clipboard

Mahsa Yarmohammadi, Shijie Wu, Marc Marone, Haoran Xu, Seth Ebner, Guanghui Qin, Yunmo Chen, Jialiang Guo, Craig Harman, Kenton Murray, Aaron Steven White, Mark Dredze, Benjamin Van Durme

A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages (2020.acl-main)

Challenge:	Zero-shot cross-lingual information extraction (IE) is a technique for training data in a source language but not in .
Approach:	They explore techniques including data projection and self-training to improve zero-shot cross-lingual information extraction (IE) IE is a construction of an IE model for some target language given existing annotations exclusively in English.
Outcome:	The proposed techniques show that they perform better than any single strategy.

Copied to clipboard

Pedro Javier Ortiz Suárez, Laurent Romary, Benoît Sagot

Adversarial Attack and Defense of Structured Prediction Models (2020.emnlp-main)

Challenge:	a recent trend in neural NLP has been the introduction of feature-based and fine-tuning methods . we train monolingual contextualized word embeddings for five mid-resource languages .
Approach:	They use common Crawl corpus to train monolingual contextualized word embeddings . they compare performance of OSCAR-based and Wikipedia-based embeddables on part-of-speech tasks .
Outcome:	The results show that OSCAR-based and Wikipedia-based embeddings perform better than Wikipedia-style embedders on part-of-speech tagging and parsing tasks.

Copied to clipboard

Wenjuan Han, Liwen Zhang, Yong Jiang, Kewei Tu

Robustness in NLP

Challenge:	Existing approaches to building effective adversarial attackers focus on classification problems.
Approach:	They propose a framework that learns to attack a structured prediction model with feedbacks from multiple reference models.
Outcome:	The proposed framework is able to attack state-of-the-art models and boost them with training . it is based on a sequence-to-sequence model with feedbacks from multiple reference models .

Rich Character-Level Information for Korean Morphological Analysis and Part-of-Speech Tagging (C18-1)

Copied to clipboard

Andrew Matteson, Chanhee Lee, Youngbum Kim, Heuiseok Lim

LinCE: A Centralized Benchmark for Linguistic Code-switching Evaluation (2020.lrec-1)

Challenge:	Korean is a highly agglutinative, character-rich language, requiring dictionary-less morphological analysis . a novel model can perform morphology and part-of-speech tagging without prior knowledge .
Approach:	They propose a multi-stage action-based model that performs morphological transformation and part-of-speech tagging using arbitrary units of input.
Outcome:	The proposed model achieves state-of-the-art word and sentence-level tagging accuracy with Korean corpus.

Copied to clipboard

Gustavo Aguilar, Sudipta Kar, Thamar Solorio

Code-Switching

Challenge:	Recent trends in NLP research have raised an interest in linguistic code-switching . however, many of these approaches are limited to a few language pairs and a specific domain .
Approach:	They propose a centralized benchmark for Linguistic Code-switching Evaluation that combines eleven corpora covering four different code-switch languages and four tasks.
Outcome:	The proposed benchmark provides a centralized benchmark and compares with other benchmarks in real-time.

Morphosyntactic Tagging with a Meta-BiLSTM Model over Context Sensitive Token Encodings (P18-1)

Copied to clipboard

Bernd Bohnet, Ryan McDonald, Gonçalo Simões, Daniel Andor, Emily Pitler, Joshua Maynez

SC-LSTM: Learning Task-Specific Representations in Multi-Task Learning for Sequence Labeling (N19-1)

Challenge:	recurrent neural networks have produced significant advances in part-of-speech tagging accuracy . a common feature of these models is the presence of rich initial word encodings . however, word or sub-word information interacts only through subsequent recursive layers .
Approach:	They propose to use recurrent neural networks with sentence-level context for initial character and word-based representations.
Outcome:	The proposed model has the highest accuracy of all participating systems in the CoNLL 2017 task.

Copied to clipboard

Peng Lu, Ting Bai, Philippe Langlais

Composing Finite State Transducers on GPUs (P18-1)

Challenge:	Multi-task learning (MTL) has been studied for sequence labeling tasks . auxiliary tasks are selected specifically to improve performance of a target task .
Approach:	They propose a shared-cell long-short-term memory cell which contains shared parameters that can learn from all tasks and task-specific parameters that could learn task-related information.
Outcome:	The proposed model can learn from all tasks and task-specific parameters.

Copied to clipboard

Arturo Argueta, David Chiang

Challenge:	Weighted finite state transducers (FSTs) are used in language processing . a GPU implementation of the composition operation is currently under development .
Approach:	They propose a GPU implementation of the composition operation for weighted finite state transducers.
Outcome:	The proposed approach achieves speedups of up to 6 times over the serial implementation and 4.5 times over OpenFST on the GPU.

VCWE: Visual Character-Enhanced Word Embeddings (N19-1)

Copied to clipboard

Chi Sun, Xipeng Qiu, Xuanjing Huang

Visual Data in NLP

Challenge:	Currently, word embeddings are playing a pivotal role in many natural language processing tasks.
Approach:	They propose a model to learn Chinese word embeddings via three-level composition . they use convolutional neural network to extract intra-character compositionality from character shape .
Outcome:	The proposed model performs better on word similarity, sentiment analysis, named entity recognition and part-of-speech tagging tasks.

Learning Better Internal Structure of Words for Sequence Labeling (D18-1)

Copied to clipboard

Yingwei Xin, Ethan Hart, Vibhuti Mahajan, Jean-David Ruvini

Representation Learning

Challenge:	a gap exists between methods for learning representations of sentences and words . authors propose a convolutional neural architecture with no down-sampling for learning words based on character embeddings .
Approach:	They propose a funnel-shaped wide convolutional neural architecture with no down-sampling for learning words' internal structure.
Outcome:	The proposed model outperforms other character embedding models on six sequence labeling datasets.

Meeting the Needs of Low-Resource Languages: The Value of Automatic Alignments via Pretrained Models (2023.eacl-main)

Copied to clipboard

Abteen Ebrahimi, Arya D. McCarthy, Arturo Oncevay, John E. Ortega, Luis Chiruzzo, Gustavo Giménez-Lugo, Rolando Coto-Solano, Katharina Kann

A multilingual collection of CoNLL-U-compatible morphological lexicons (L18-1)

Challenge:	Large multilingual models have inspired a new class of word alignment methods, which work well for pretraining languages.
Approach:	They propose to use transformer-based word alignment methods to extract alignments from massive pretrained models.
Outcome:	The proposed methods outperform traditional methods for languages unseen to pretraining models, and are competitive with each other.

Copied to clipboard

Benoît Sagot

Substructure Substitution: Structured Data Augmentation for NLP (2021.findings-acl)

Challenge:	Existing morphological lexicons are limited in scope and are not universally accepted . morphology lexical information is encoded into morphologists or gathered in lexiconics .
Approach:	They propose a multilingual collection of morphological lexicons that follow the Universal Dependencies initiative.
Outcome:	The proposed collection of 53 morphological lexicons covers 38 languages . they have been shown to improve part-of-speech tagging and parsing accuracy .

Copied to clipboard

Haoyue Shi, Karen Livescu, Kevin Gimpel

Multimodality

Challenge:	Existing work focuses on word-level manipulation or global sequence-to-sequence style generation.
Approach:	They propose a family of data augmentation methods that generalize prior methods by substituting substructures with others having the same label.
Outcome:	The proposed methods can be applied to many structured NLP tasks such as part-of-speech tagging and parsing.

Data-Efficient French Language Modeling with CamemBERTa (2023.findings-acl)

Copied to clipboard

Wissam Antoun, Benoît Sagot, Djamé Seddah

Increasing Learning Efficiency of Self-Attention Networks through Direct Position Interactions, Learnable Temperature, and Convoluted Attention (2020.coling-main)

Challenge:	Recent advances in NLP have significantly improved the performance of language models on a variety of tasks.
Approach:	They introduce a French DeBERTa model that builds upon the DeBERTAV3 architecture and training objective and evaluate its performance on a variety of French downstream tasks and datasets.
Outcome:	The proposed model outperforms BERT-based models on most tasks given the same amount of training tokens and trained on 30% of its input tokens.

Copied to clipboard

Philipp Dufter, Martin Schmitt, Hinrich Schütze

Women’s Syntactic Resilience and Men’s Grammatical Luck: Gender-Bias in Part-of-Speech Tagging and Dependency Parsing (P19-1)

Challenge:	SANs are an integral part of successful neural networks such as Transformer . training SAN on a task or pretraining them on language modeling requires large amounts of data and compute resources.
Approach:	They propose to modify SANs to enable faster learning, i.e., higher accuracies after fewer update steps.
Outcome:	The proposed modifications enable faster learning, i.e., higher accuracies after fewer update steps.

Copied to clipboard

Aparna Garimella, Carmen Banea, Dirk Hovy, Rada Mihalcea

How to Adapt Your Pretrained Multilingual Model to 1600 Languages (2021.acl-long)

Challenge:	linguistic studies have shown the prevalence of various lexical and grammatical patterns in texts authored by a person of a particular gender, but models for part-of-speech tagging and dependency parsing have not adapted to account for these differences.
Approach:	They annotate the Wall Street Journal part of the Penn Treebank with the gender information of the articles’ authors and build taggers and parsers trained on this data.
Outcome:	The proposed model can account for gendered differences in syntactic tasks and highlight future venues for developing more accurate taggers and parsers.

Copied to clipboard

Abteen Ebrahimi, Katharina Kann

Prompting Language Models for Linguistic Structure (2023.acl-long)

Challenge:	Pretrained multilingual models perform best for languages seen during pretraining . methods exist to improve performance for unseen languages, but have been evaluated using amounts of raw text only available for a small fraction of the world’s languages.
Approach:	They evaluate the performance of existing methods to adapt pretrained multilingual models to new languages using a resource available for close to 1600 languages: the New Testament.
Outcome:	The proposed models perform best for languages seen during pretraining . the results show that the most efficient approach is simplest and the most accurate .

Copied to clipboard

Terra Blevins, Hila Gonen, Luke Zettlemoyer

Distillation-Resistant Watermarking for Model Protection in NLP (2022.findings-emnlp)

Challenge:	Existing prompting methods can test this hypothesis on autoregressive PLMs.
Approach:	They propose a structured prompting approach for linguistic structured prediction tasks that performs zero- and few-shot sequence tagging with autoregressive PLMs.
Outcome:	The proposed approach shows that the model can perform few-shot sequence tagging on part-of-speech taging, named entity recognition, and sentence chunking tasks.

Copied to clipboard

Xuandong Zhao, Lei Li, Yu-Xiang Wang

Small and Practical BERT Models for Sequence Labeling (D19-1)

Challenge:	Existing protection methods such as watermarking only work for images but are not applicable to text.
Approach:	They propose a technique that injects watermarks into the victim’s prediction probability corresponding to a secret key and is able to detect such a key by probing a suspect model.
Outcome:	The proposed technique detects stealing suspects at 100% accuracy on four NLP tasks while the prior method fails on two.

Copied to clipboard

Henry Tsai, Jason Riesa, Melvin Johnson, Naveen Arivazhagan, Xin Li, Amelia Archer

An Empirical Study of Pre-trained Transformers for Arabic Information Extraction (2020.emnlp-main)

Challenge:	Existing models for morphosyntactic tagging have focused on building separate models for each language or for a small group of related languages.
Approach:	They propose a scheme to train a single multilingual sequence labeling model that is small and fast enough to run on a CPU.
Outcome:	The proposed model outperforms state-of-the-art models on low-resource languages and low-level models on codemixed inputs.

Copied to clipboard

Wuwei Lan, Yang Chen, Wei Xu, Alan Ritter

What’s so special about BERT’s layers? A closer look at the NLP pipeline in monolingual and multilingual models (2020.findings-emnlp)

Challenge:	Multilingual pre-trained Transformers have been shown to enable effective cross-lingual zero-shot transfer, but their performance on Arabic information extraction tasks is not well studied.
Approach:	They pre-train a bilingual BERT that is designed specifically for Arabic NLP and English-to-Arabic zero-shot transfer learning.
Outcome:	The pre-trained model significantly outperforms mBERT, XLM-RoBERTa, and AraBERT in both the supervised and zero-shot transfer settings.

Copied to clipboard

Wietse de Vries, Andreas van Cranenburgh, Malvina Nissim

OFrLex: A Computational Morphological and Syntactic Lexicon for Old French (2020.lrec-1)

Challenge:	In addition, information on part-of-speech tagging is spread over different parts of the network and the pipeline might not be as neat as it seems.
Approach:	They propose to probe Dutch BERT-based model and multilingual BERT model for Dutch NLP tasks to see if this holds true for other languages.
Outcome:	The proposed model is based on a Dutch model and a multilingual model for Dutch NLP tasks.

Copied to clipboard

Gaël Guibon, Benoît Sagot

Building an Endangered Language Resource in the Classroom: Universal Dependencies for Kakataibo (2022.lrec-1)

Challenge:	Using heterogeneous language resources, we extract structured and exploitable information from a large-coverage morphological and syntactic Old French lexicon.
Approach:	They propose to use a large-coverage morphological and syntactic Old French lexicon to extract structured and exploitable information from heterogeneous language resources.
Outcome:	The proposed extension technique will be validated manually in the near future and take advantage of OFrLex’s viewing, searching and editing interface.

Copied to clipboard

Roberto Zariquiey, Claudia Alvarado, Ximena Echevarría, Luisa Gomez, Rosa Gonzales, Mariana Illescas, Sabina Oporto, Frederic Blum, Arturo Oncevay, Javier Vera

MAD-G: Multilingual Adapter Generation for Efficient Cross-Lingual Transfer (2021.findings-emnlp)

Challenge:	12 This paper describes the collaborative methodology implemented to create a UD treebank for a Peruvian endangered language.
Approach:	They propose to create a UD treebank for a Peruvian endangered language . they use a collaborative methodology to create the treebank in a course .
Outcome:	The proposed treebank would enhance the future development of an NLP toolkit for this endangered language.

Copied to clipboard

Alan Ansell, Edoardo Maria Ponti, Jonas Pfeiffer, Sebastian Ruder, Goran Glavaš, Ivan Vulić, Anna Korhonen

Lexically Grounded Subword Segmentation (2024.emnlp-main)

Challenge:	Massively multilingual transformers (MMTs) have benefited from additional training of language-specific adapters, but this approach is not viable for the vast majority of languages due to limitations in their corpus size or compute budgets.
Approach:	They propose a multilingual ADapter generation approach which contextually generates language adapters from language representations based on typological features.
Outcome:	The proposed method improves cross-lingual transfer performance on part-of-speech tagging, dependency parsing, and named entity recognition tasks while remaining cost-effective.

Copied to clipboard

Jindřich Libovický, Jindřich Helcl

Evaluating Pixel Language Models on Non-Standardized Languages (2025.coling-main)

Challenge:	Statistical word segmentation algorithms have remained a thorn in the side of many researchers.
Approach:	They propose to use unsupervised morphological analysis with Morfessor as pre-tokenization and an algebraic method for obtaining subword embeddings grounded in a word embeddable space.
Outcome:	The proposed methods improve morphological plausibility and Rényi efficiency on part-of-speech tagging and machine translation tasks.

Copied to clipboard

Alberto Muñoz-Ortiz, Verena Blaschke, Barbara Plank

A Warm Start and a Clean Crawled Corpus - A Recipe for Good Language Models (2022.lrec-1)

Challenge:	pixel-based models can be used to transfer learning from standard languages to dialects . pretrained language models achieve strong results for languages seen during training, but their performance declines with out-of-domain dialects.
Approach:	They compare pixel-based models to token-based ones to evaluate dialects . standard german is tokenized in a more meaningful way, whereas the Bern dialect is tokenize in pixel form .
Outcome:	The proposed models outperform token-based models in part-of-speech tagging, dependency parsing and intent detection for zero-shot dialect evaluation by up to 26 percentage points in some scenarios, though not in Standard German.

Copied to clipboard

Vésteinn Snæbjarnarson, Haukur Barri Símonarson, Pétur Orri Ragnarsson, Svanhvít Lilja Ingólfsdóttir, Haukur Jónsson, Vilhjalmur Thorsteinsson, Hafsteinn Einarsson

DrBenchmark: A Large Language Understanding Evaluation Benchmark for French Biomedical Domain (2024.lrec-main)

Challenge:	Pre-trained neural language models have shown impressive results when adapted for a variety of classification and text generation tasks.
Approach:	They propose to use Icelandic's Icelandic Common Crawl Corpus to train language models that achieve state-of-the-art performance in downstream tasks.
Outcome:	The proposed models achieve state-of-the-art in a variety of downstream tasks including part-of speech tagging, named entity recognition and constituency parsing.

Copied to clipboard

Yanis Labrak, Adrien Bazoge, Oumaima El Khettari, Mickael Rouvier, Pacome Constant Dit Beaufils, Natalia Grabar, Béatrice Daille, Solen Quiniou, Emmanuel Morin, Pierre-Antoine Gourraud, Richard Dufour

Kvistur 2.0: a BiLSTM Compound Splitter for Icelandic (2020.lrec-1)

Challenge:	Existing benchmarks for pre-trained language models are limited to only a few languages . a limited number of tasks are evaluated on non-standardized protocols .
Approach:	They propose to aggregate diverse downstream tasks into a benchmark to assess PLMs' qualities . they evaluate 8 pre-trained masked language models on general and biomedical-specific data .
Outcome:	The proposed benchmark assesses pre-trained language models on 20 diversified tasks.

Copied to clipboard

Jón Daðason, David Mollberg, Hrafn Loftsson, Kristín Bjarnadóttir

Multi-Source Domain Adaptation with Mixture of Experts (D18-1)

Challenge:	Compounding is highly productive in Icelandic, and new compounds are constantly being created.
Approach:	They propose a character-based biLSTM model for splitting Icelandic compound words . the model learns how to split compound words into two parts .
Outcome:	The proposed model outperforms other methods on a corpus of manually split word forms.

Copied to clipboard

Jiang Guo, Darsh Shah, Regina Barzilay

Low-Resource NLP

Challenge:	Existing methods for domain adaptation from multiple sources are designed to transfer supervision from a single source domain.
Approach:	They propose to capture the relationship between a target example and different source domains by a point-to-set metric.
Outcome:	The proposed method outperforms baselines and can handle negative transfer.

What can we learn from Semantic Tagging? (D18-1)

Copied to clipboard

Mostafa Abdou, Artur Kulmizev, Vinit Ravishankar, Lasha Abzianidze, Johan Bos

Make the Best of Cross-lingual Transfer: Evidence from POS Tagging with over 100 Languages (2022.acl-long)

Challenge:	a recent study shows that multi-task learning improves performance of NLP tasks by exploiting similarities between tasks.
Approach:	They employ semantic tagging as an auxiliary task for three NLP tasks . they compare full neural network sharing, partial neural network shared and learning what to share .
Outcome:	The proposed model improves for part-of-speech tagging, universal dependency parsing and natural language inference.

Copied to clipboard

Wietse de Vries, Martijn Wieling, Malvina Nissim

Low-Resource NLP

Challenge:	Existing studies on cross-lingual generalisability of large pre-trained models use English training data and test data in unseen languages.
Approach:	They propose to use multilingual pre-trained models to model cross-lingual transfer in a selection of target languages.
Outcome:	The proposed model can be used to improve cross-lingual transfer performance in low-resource languages with no labeled training data.

SlovakBERT: Slovak Masked Language Model (2022.findings-emnlp)

Copied to clipboard

Matúš Pikuliak, Štefan Grivalský, Martin Konôpka, Miroslav Blšták, Martin Tamajka, Viktor Bachratý, Marian Simko, Pavol Balážik, Michal Trnka, Filip Uhlárik

LemmaTag: Jointly Tagging and Lemmatizing for Morphologically Rich Languages with BRNNs (D18-1)

Challenge:	SlovakBERT is a new masked language model that is based on a Web-crawled corpus.
Approach:	They introduce a new Slovak-only transformers-based language model called SlovkBERT . they evaluate the model on several NLP tasks and establish a benchmark for Slovakia .
Outcome:	The proposed model achieves state-of-the-art on several NLP tasks and achieves best results . the proposed model could be used by other Slovak researchers or NLP practitioners .

Copied to clipboard

Daniel Kondratyuk, Tomáš Gavenčiak, Milan Straka, Jan Hajič

Linguistically-Informed Self-Attention for Semantic Role Labeling (D18-1)

Challenge:	We compare morphologically rich languages with analytical languages like English due to the large vocabulary size and data sparsity.
Approach:	They propose a featureless neural network architecture that generates part-of-speech tags and lemmas for sentences by using bidirectional RNNs with character-level and word-level embeddings.
Outcome:	The proposed model outperforms state-of-the-art models in Czech, German, and Arabic.

Copied to clipboard

Emma Strubell, Patrick Verga, Daniel Andor, David Weiss, Andrew McCallum

Exploring the Language of Data (2020.coling-main)

Challenge:	Existing models of semantic role labeling use no explicit linguistic features. prior work has shown that syntax trees can dramatically improve SRL decoding.
Approach:	They propose a neural network model that incorporates syntax using only raw tokens . they show that LISA out-performs the state-of-the-art with contextually-encoded word representations a 1.0 F1 on newswire and 2.0 F1 in out-of domain text .
Outcome:	The proposed model outperforms the state-of-the-art model with word embeddings and predicted predicates.

Copied to clipboard

Gábor Bella, Linda Gremes, Fausto Giunchiglia

Multimodality

Challenge:	Structured data, such as database tables or XML trees, often contain short natural language labels that describe the data structure itself or provide content (attribute values). Conventional NLP tools, such supervised sequence labellers or embeddings trained on full sentences, do not perform well on structured data.
Approach:	They propose to design a type of abbreviated grammar that is called the Language of Data and to investigate the grammatical properties of such labels.
Outcome:	The proposed model outperforms models trained on standard text on tokenisation, part-of-speech tagging, and named entity recognition over real-world structured data.

Exploring and Predicting Transferability across NLP Tasks (2020.emnlp-main)

Copied to clipboard

Tu Vu, Tong Wang, Tsendsuren Munkhdalai, Alessandro Sordoni, Adam Trischler, Andrew Mattarella-Micke, Subhransu Maji, Mohit Iyyer

CamemBERT: a Tasty French Language Model (2020.acl-main)

Challenge:	Recent advances in NLP demonstrate the effectiveness of training large-scale language models and transferring them to downstream tasks.
Approach:	They conduct an extensive study of the transferability between 33 NLP tasks across three broad classes of problems.
Outcome:	The proposed model can improve performance even with low-data source tasks that differ substantially from the target task.

Copied to clipboard

Louis Martin, Benjamin Muller, Pedro Javier Ortiz Suárez, Yoann Dupont, Laurent Romary, Éric de la Clergerie, Djamé Seddah, Benoît Sagot

Sparsity Makes Sense: Word Sense Disambiguation Using Sparse Contextualized Word Representations (2020.emnlp-main)

Challenge:	Pretrained language models are now ubiquitous in Natural Language Processing, but their use in other languages is limited.
Approach:	They propose to train monolingual Transformer-based model for other languages using web crawled data instead of Wikipedia data and a relatively small web crawl dataset leads to better results.
Outcome:	The proposed model performs as well as those obtained using larger datasets.

Copied to clipboard

Gábor Berend

PyCantonese: Cantonese Linguistics and NLP in Python (2022.lrec-1)

Challenge:	Using sparse word embeddings is highly applicable for word sense disambiguation (WSD) .
Approach:	They propose an overcomplete set of semantic basis vectors that allows for sparse word representations.
Outcome:	The proposed framework achieves an aggregated F score of 78.8 over five standard word sense disambiguating benchmark datasets.

Copied to clipboard

Jackson Lee, Litong Chen, Charles Lam, Chaak Ming Lau, Tsz-Him Tsui

Prefix Text as a Yarn: Eliciting Non-English Alignment in Foundation Language Model (2024.findings-acl)

Challenge:	a limited number of Cantonese-specific datasets are available for PyCantones.
Approach:	They introduce PyCantonese, an open-source Python library for Cantonesi linguistics and natural language processing.
Outcome:	The proposed library is open-source and available for free for all purposes, including commercial ones.

Copied to clipboard

Runzhe Zhan, Xinyi Yang, Derek Wong, Lidia Chao, Yue Zhang

A Multitask Learning Approach for Diacritic Restoration (2020.acl-main)

Challenge:	supervised fine-tuning (SFT) has been a straightforward approach for tailoring the output of foundation large language models (LLMs) to specific preferences.
Approach:	They propose a training-free alignment method that uses minimal prior tokens to bridge the foundation LLM and the SFT LLM.
Outcome:	The proposed method achieves comparable performance without training on machine translation and part-of-speech tagging across seven languages.

Copied to clipboard

Sawsan Alqahtani, Ajay Mishra, Mona Diab

Challenge:	Diacritics are used to specify pronunciations and meanings in many languages like Arabic.
Approach:	They propose to use multi-task learning to optimize diacritic restoration with related NLP problems . they use Arabic as a case study since it has sufficient data resources for tasks .
Outcome:	The proposed model outperforms baseline models and is comparable to the state-of-the-art models.

NLPre: A Revised Approach towards Language-centric Benchmarking of Natural Language Preprocessing Systems (2024.lrec-main)

Copied to clipboard

Martyna Wiącek, Piotr Rybak, Łukasz Pszenny, Alina Wróblewska

Model-Based Ranking of Source Languages for Zero-Shot Cross-Lingual Transfer (2025.emnlp-main)

Challenge:	GLUE benchmarking system enables ongoing evaluation of multiple NLPre tools while credibly tracking their performance.
Approach:	They propose a language-centric benchmarking system that enables ongoing evaluation of multiple NLPre tools while credibly tracking their performance.
Outcome:	The proposed system is configured for Polish and integrated with the thoroughly assembled NLPre-PL benchmark.

Copied to clipboard

Abteen Ebrahimi, Adam Wiemerslage, Katharina von der Wense